Token-based billing model for server-side rendering service

ABSTRACT

A token-based billing model for delivering server-side rendered applications to remote users. A token represents a right to access a server-side rendered application. Each remote user is associated with one or more tokens. When a given token expires, the set of tokens associated with a user is decremented. The rate at which tokens expire are modulated based on aggregate load across the resources of the server-side rendered application provider, as well as the individual loads attributable to each of the users.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/930,113, filed Nov. 2, 2015, entitled“TOKEN-BASED BILLING MODEL FOR SERVER-SIDE RENDERING SERVICE” whichclaims priority to U.S. patent application Ser. No. 12/964,153, filedDec. 9, 2010, entitled “TOKEN-BASED BILLING MODEL FOR SERVER-SIDERENDERING SERVICE”, which itself claims priority to Prov. U.S. Pat. App.Ser. No. 61/285,283, filed Dec. 10, 2009, entitled “TOKEN-BASED BILLINGMODEL FOR SERVER-SIDE RENDERING SERVICE”, all of which are incorporatedherein by reference in their entireties, for all purposes.

TECHNICAL FIELD

The present disclosure generally relates to server-side renderedapplication services and, more particularly, to a token-based billingmodel and system for the delivery of server-side rendered applications,such as video games, office or productivity applications, and the like.

BACKGROUND

Server-side rendering generally refers to the concept of processing andrendering computer graphics, e.g. video sequences, digital images ortext, on computer servers and delivering the resulting graphical outputto remote users at their client devices. In one form, the graphicaloutput is delivered as a series of video frames compressed using a videocodec. One of the popular applications for server-side rendering iscomputer games, which often place high demands on the graphicalresources of the computer systems used for game execution. Instead offorcing a game player to choose between investing a great deal of moneyon a high-end computer system capable of satisfying the demands placedon the graphical resources by sophisticated computer games and acceptinglower quality graphics, sever-side rendering provides an alternativesolution. A game player may choose to have a computer game rendered on aserver system, which often contains better resources, and delivered to aremote client device as a video sequence usually compressed using anegotiated video codec. In this case, the client device only needs todecompress and display the processed and rendered images.

Of course, computer games are not the only applications for whichserver-side rendering is suitable. For example, with video streaming,individual frames of a video may be decoded and rendered on a serversystem and delivered to the client devices. Similarly, videos and imagesare not the only types of visual results that may be rendered on aserver system. In fact, server-side rendering may be used to render anytype of application. For example, text-based applications, such as wordprocessors and spreadsheets, may be rendered on a computer server anddelivered to the client devices as well, since, similar to digitalimages, since the output of such applications is represented as pixelson the display systems.

SUMMARY

The present disclosure generally relates to server-side renderedapplication services and, more particularly, to a token-based billingmodel for delivering server-side rendered applications to remote users.

In particular embodiments, a token represents a right to access aserver-side rendered application. Each remote user is associated withone or more tokens. When a given token expires, the set of tokensassociated with a user is decremented. When all tokens are exhausted, agiven user must acquire more tokens for access to the server-siderendered application. The rate at which tokens expire, in oneimplementation, can initially be set to a base expiration rate, and thenmodulated based on aggregate load across the resources of theserver-side rendered application provider, as well as the individualloads attributable to each of the users.

These and other features, aspects, and advantages of the disclosure aredescribed in more detail below in the detailed description and inconjunction with the following figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 illustrates an example system for providing and charging forserver-side rendering services.

FIG. 2 illustrates an example computer system architecture.

FIG. 3 is a state diagram associated with a token-based billing modelfor server-side rendered application services.

FIG. 4 illustrates an example method for adjusting a token expirationrate for server-side rendering services.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The present disclosure is now described in detail with reference to afew example embodiments thereof as illustrated in the accompanyingdrawings. In the following description, numerous specific details areset forth in order to provide a thorough understanding of the presentdisclosure. It is apparent, however, to one skilled in the art, that thepresent disclosure may be practiced without some or all of thesespecific details. In other instances, well known process steps and/orstructures have not been described in detail in order to notunnecessarily obscure the present disclosure. In addition, while thedisclosure is described in conjunction with the particular embodiments,it should be understood that this description is not intended to limitthe disclosure to the described embodiments. To the contrary, thedescription is intended to cover alternatives, modifications, andequivalents as may be included within the spirit and scope of thedisclosure as defined by the appended claims.

Overview

In particular embodiments, a server-side application rendering systemdelivers, utilizing one or more servers, a server-side renderedapplication to one or more remote users. Each of the remote users isassociated with a set of tokens, where each token represents a right toaccess the server-side rendered application. The server-side applicationrendering system modulates a rate at which tokens expire based on loadobserved at the one or more servers caused by delivering the server-siderendered application. In some implementations, the server-sideapplication rendering system may evaluate the individual load associatedwith delivering the server-side rendered application to each remoteuser. Still further, the server-side application rendering system maysupport user interface controls that allow a remote user to adjust atleast one session quality parameter that affects the relative loadassociated with delivering the server-side rendered application to theremote user, therefore adjusting the rate at which tokens expire.

The load associated with delivering a rendered application to a remoteuser can be assessed based on both the computing and graphics resourcesrequired to host the application, render the application output, and/orthe bandwidth that is consumed to transmit the rendered output to theremote user. In particular embodiments, the first set of factors thatrelate to the load attributable to executing and rendering of theapplication may include, but is not limited to, the quality of theimage, the number of pixels contained in the image, the processingoperations performed on the image including three-dimensional (3D) andtwo-dimensional (2D) graphics operations, the amount of rendering donefor the images, the amount of resources used for rendering the image,the time the rendering of the image is performed, etc. In particularembodiments, the second set of factors that relate to the loadassociated with delivering the rendered output may include, but is notlimited to, the bandwidth and latency of the transmission, thecompression ratio, the encryption applied to the image, etc.

Server-Side Rendering

FIG. 1 illustrates an example network environment in which particularimplementations of the invention may operate. As FIG. 1 illustrates,particular implementations of the invention may operate in a networkenvironment comprising a video transmission system 20 that isoperatively coupled to a network cloud 60, which may include theInternet. Network cloud 60 generally represents one or moreinterconnected networks, over which the systems and hosts describedherein can communicate. Network cloud 60 may include packet-based widearea networks (such as the Internet), private networks, wirelessnetworks, satellite networks, cellular networks, paging networks, andthe like. Some of the networks in network cloud 60 may becircuit-switched networks. The computer network environment, includingnetwork 60 can be a packet-based communications environment, employingTCP/IP protocols (for example), and/or other suitable protocols, and hasa plurality of interconnected digital packet transmission stations orrouting nodes. Client nodes 82 and 84 are operably connected to thenetwork environment via a network service provider or any other suitablemeans.

Client nodes 82 and 84 may include personal computers or cell phones, aswell as other types of mobile or portable devices such as laptopcomputers, netbooks, personal digital assistants (PDAs), etc. One ormore links couple each client 82, 84 and server-side applicationrendering system 20 to network 60. In particular embodiments, one ormore links each includes one or more wireline, wireless, cellular oroptical links. In particular embodiments, one or more links eachincludes an intranet, an extranet, a virtual private network (VPN), aLAN, a WLAN, a WAN, a MAN, a portion of the Internet, or another link ora combination of two or more such links. The present disclosurecontemplates any suitable links coupling clients 82, 84 and server-sideapplication rendering system 20 to network 60.

Server-side application rendering system 20 is a network addressablesystem that hosts one or more applications accessible to one or moreusers over a computer network. Server-side application rendering system20 may include web site and server functionality where users may requestand receive identified web pages, video streams, applications and othercontent over the computer network. In particular implementations,server-side application rendering system 20 comprises one or morephysical servers 22 and one or more data stores 24. The one or morephysical servers 22 are operably connected to computer network 60 via arouter 26. The one or more physical servers 22 host functionality thatallows users to browse available content, such as receiving requestsfrom, and transmitting responsive data to, client devices 82, 84. In oneimplementation, the functionality hosted by the one or more physicalservers may include web or HTTP servers, RTSP servers, and the like.

Physical servers 22, as discussed above, host functionality directed tosupporting and implementing server-side application rendering system 20.In a particular implementation, the physical servers 22 may host one ormore applications (such as a video game, a word processing program, andthe like), as well as video rendering, compression and streamingfunctionality. In one implementation, a data store 24 may store videocontent such as digital content data objects, application code, dataobjects, user account information, and media assets. A content dataobject or a content object, in particular implementations, is anindividual item of digital information typically stored or embodied in adata file or record. Content objects may take many forms, including:text (e.g., ASCII, SGML, HTML), images (e.g., jpeg, tif and gif),graphics (vector-based or bitmap), audio, video (e.g., mpeg), or othermultimedia, and combinations thereof. Content object data may alsoinclude executable code objects, object or asset definitions, etc.Structurally, content data store 24 connotes a large class of datastorage and management systems. In particular implementations, contentdata store 24 may be implemented by any suitable physical systemincluding components, such as database servers, mass storage media,media library systems, and the like.

The server and client host systems described herein may be implementedin a wide array of computing systems and architectures. The followingdescribes example computing architectures for didactic, rather thanlimiting, purposes. FIG. 2 illustrates an example computing systemarchitecture, which may be used to implement a physical server and, insome instances, a client host. In one embodiment, hardware system 200comprises a processor 202, a cache memory 204, and one or more softwareapplications and drivers directed to the functions described herein.Additionally, hardware system 200 includes a high performanceinput/output (I/O) bus 206 and a standard I/O bus 208. A host bridge 210couples processor 202 to high performance I/O bus 206, whereas I/O busbridge 212 couples the two buses 206 and 208 to each other. A systemmemory 214 and a network/communication interface 216 couple to bus 206.For physical servers and clients hosting video compressionfunctionality, hardware system 200 may further include one or moregraphics processing units 224 coupled to buses 206 and 208. In oneimplementation, the graphics processing unit 224 may be embodied in agraphics or display card that attaches to the hardware systemarchitecture via a card slot. In other implementations, the graphicsprocessor unit 224 may be integrated on the motherboard of the serversystem architecture. Suitable graphics processing units include AdvancedMicro Devices®AMD R7XX based GPU devices (Radeon® HD 4XXX), AMD R8XXbased GPU devices (Radeon® HD 5XXX), Intel® Larabee based GPU devices(yet to be released), nVidia® 8000 series GPUs, nVidia® 9000 seriesGPUs, nVidia® GF100 series GPUs, nVidia® 200 series GPUs, and any otherDX11-capable GPUs.

Mass storage 218, and I/O ports 220 couple to bus 208. Hardware system200 may optionally include a keyboard and pointing device, and a displaydevice (not shown) coupled to bus 208. Collectively, these elements areintended to represent a broad category of computer hardware systems,including but not limited to general purpose computer systems based onthe x86-compatible processors manufactured by Intel Corporation of SantaClara, California, and the x86-compatible processors manufactured byAdvanced Micro Devices (AMD), Inc., of Sunnyvale, California, as well asany other suitable processor.

The elements of hardware system 200 are described in greater detailbelow. In particular, network interface 216 provides communicationbetween hardware system 200 and any of a wide range of networks, such asan Ethernet (e.g., IEEE 802.3) network, etc. Mass storage 218 providespermanent storage for the data and programming instructions to performthe above described functions implemented in the location server 22,whereas system memory 214 (e.g., DRAM) provides temporary storage forthe data and programming instructions when executed by processor 202.I/O ports 220 are one or more serial and/or parallel communication portsthat provide communication between additional peripheral devices, whichmay be coupled to hardware system 200.

Hardware system 200 may include a variety of system architectures; andvarious components of hardware system 200 may be rearranged. Forexample, cache 204 may be on-chip with processor 202. Alternatively,cache 204 and processor 202 may be packed together as a “processormodule,” with processor 202 being referred to as the “processor core.”Furthermore, certain embodiments of the present invention may notrequire nor include all of the above components. For example, theperipheral devices shown coupled to standard I/O bus 208 may couple tohigh performance I/O bus 206. In addition, in some embodiments, only asingle bus may exist, with the components of hardware system 200 beingcoupled to the single bus. Furthermore, hardware system 200 may includeadditional components, such as additional processors, storage devices,or memories.

Graphics processing unit 224, in one implementation, comprises one ormore integrated circuits and/or processing cores that are directed tomathematical operations commonly used in graphics rendering. In someimplementations, the GPU 224 may use a special graphics unit instructionset, while in other implementations, the GPU may use a CPU-like (e.g. amodified x86) instruction set. Graphics processing unit 224 canimplement a number of graphics primitive operations, such as Miffing,texture mapping, pixel shading, frame buffering, and the like. Inaddition to the 3D hardware, graphics processing unit 224 may includebasic 2D acceleration and frame buffer capabilities. In addition,graphics processing unit 224 may support the YUV color space andhardware overlays, as well as MPEG primitives (such as motioncompensation and iDCT). Graphics processing unit 224 may be a graphicsaccelerator, a GPGPU (General Purpose GPU), or any other suitableprocessing unit.

As discussed below, in one implementation, the operations of one or moreof the physical servers described herein are implemented as a series ofsoftware routines run by hardware system 200. These software routinescomprise a plurality or series of instructions to be executed by aprocessor in a hardware system, such as processor 202. Initially, theseries of instructions may be stored on a storage device or othercomputer readable medium, such as mass storage 218. However, the seriesof instructions can be stored on any suitable storage medium, such as adiskette, CD-ROM, ROM, EEPROM, etc. Furthermore, the series ofinstructions need not be stored locally, and could be received from aremote storage device, such as a server on a network, vianetwork/communication interface 216. The instructions are copied fromthe storage device, such as mass storage 218, into memory 214 and thenaccessed and executed by processor 202. The software routines can causecertain operations to be performed by the graphics processing unit 224and/or the processor 202.

An operating system manages and controls the operation of hardwaresystem 200, including the input and output of data to and from softwareapplications (not shown). The operating system provides an interfacebetween the software applications being executed on the system and thehardware components of the system. According to one embodiment of thepresent invention, the operating system is the Windows®95/98/NT/XP/Vista/7 operating system, available from MicrosoftCorporation of Redmond, Wash. However, the present invention may be usedwith other suitable operating systems, such as the Apple MacintoshOperating System, available from Apple Computer Inc. of Cupertino,Calif., UNIX operating systems, LINUX operating systems, and the like.Of course, other implementations are possible. For example, the serverfunctionalities described herein may be implemented by a plurality ofserver blades communicating over a backplane.

In a particular implementation, a server 22 is configured to host one ormore applications the rendered output of which is transmitted to remoteusers over respective user sessions. In one implementation, the server22 hosts an instance (sometimes called a thread) of a given applicationfor a given user session, generating rendered output utilizing the GPUand transmitting the rendered output as a video stream to the remoteuser. Therefore, the computing and graphics processing resources of aserver 22 may be shared among multiple users. In one implementation, agiven server 22 may include multiple GPUs to serve multiple usersconcurrently. In addition, the resources of a single GPU may be sharedamong multiple client devices/remote users. In one implementation, therendered output of an instance of an application comprises a sequence ofvideo frames. These video frames can be streamed to a client 82, 84. Inone implementation, a video codec processes video frames buffered in amemory. The memory may be main CPU memory or, in some implementations,the memory buffers available on a GPU. For example, a separate renderingprocess executing on a GPU (or a CPU) may render the video frame. Theexample codec may process the frame as described below for output to avideo client for decoding and display.

Additionally, video decoding clients may be hosted on the same orsimilar hardware architectures. For example, client computer 82 mayinclude a GPU which loads encoded video into GPU memory, and decodes theencoded video data to render one or more frames of a video sequence. Inother implementations, some or all of the video compression andde-compression operations may also be performed in system memory using ageneral purpose computing architecture as well.

Token-Based Billing Model

In particular embodiments, a token represents a right to access aserver-side rendered application and, more generally, the resources ofserver-side application rendering system 20. Each remote user isassociated with one or more tokens. When a given token expires, the setof tokens associated with a user is decremented. When all tokens areexhausted, a given user must acquire more tokens for access to theserver-side application rendering system 20. The rate at which tokensexpire, in one implementation, can initially be set to a base expirationrate, and then modulated based on aggregate load across the resources ofthe server-side rendered application system, as well as the individualloads attributable to each of the users.

A token may be implemented in a variety of manners. In oneimplementation, a token may be represented as a unit amount associatedwith a user account, where the number of tokens associated with a useris expressed as a parameter value in a corresponding field of a useraccount record. In some implementations, a token or set of tokens may berepresented as a digital object that has been digitally signed using acryptographic function, such as SHA and MD5. For example, a token dataobject may include a message and a message digest. The message mayinclude a variety of fields, such as a user account identifier, anidentifier of the token issuing entity, a number of tokens, time stamps,the application types for which the token can be used, and the like. Themessage digest is a hashed value of the message. The token data objectmay be stored locally with a given client application (and passed in abrowser cookie or other state object) and/or remotely in a remotedatabase. An individual or entity may purchase or otherwise acquiretokens for itself or for transfer to others. For example, a user maypurchase a set of tokens in order to access a network application, suchas a video game, supported by a server-side application rendering system20. In some implementations, an entity may acquire tokens from theserver-side application rendering system 20 and transfer them to usersas part of a promotional or marketing campaign.

In a particular implementation, a token expires at a base expirationrate, entitling a given user to a certain unit amount of access to thecomputing resources of the server-side application rendering system 20.For example, if the expiration rate does not change from the defaultrate, a token may allow a user to access the resources of applicationrender farm for a given amount of time, such as 30 minutes. As discussedbelow, however, the rate at which tokens expire may vary dynamicallybased on observed loading conditions. Even the base expiration rate mayvary depending on the type of access during a given user session. Forexample, accessing using a mobile device with a smaller screen sizetypically requires less computing resources—especially GPU resources;therefore, the base rate for the session may be lower relative to thebase rate for a session where a user accesses a network applicationrendered by the application render farm using a personal computer andrequests a larger window size, such as the entire display screen (e.g.,1920×1080 pixels) and rendering at full resolution.

In one implementation, the base expiration rate for a given user sessionmay be based on the number of render units allocated to the usersession. A render unit, in one implementation, represents a fixed pixelarea, such as 256×256 or 128×128 pixels allocated to a given usersession. As discussed above, the resources of a server 22 include atleast one GPU that includes a render target (a defined memory space,such as a frame buffer) that supports a vast array of pixels. Thisrender target and the remaining facilities of the GPU can be used forone to many user sessions. Accordingly, the render target can beconceptually divided into render units and allocated to different usersessions. The number of render units allocated to a given user sessionmay depend on the type of client device (and corresponding displaytype), as well as the resolution at which the application is to berendered. The display systems of client devices, such as laptops,desktops, notebooks and mobile devices can vary considerably. Forexample, the iPhone® offered by Apple, Inc.® of Cupertino, Californiahas a 3.5-inch (diagonal) display screen with a pixel resolution of480×320 pixels. Laptops and desktop computers may have considerablylarger displays. For example, a desktop computer may have a 21-inch(diagonal) display with a 1920-by-1080 pixel resolution. Accordingly, auser session associated with a desktop client may be allocated enoughrender units to account for the desired display size. Still further, thegreater number of pixels to render requires more bandwidth relative touser sessions with lower number of pixels. In other implementations, thebase expiration rate does not depend on the number of render unitsallocated to a given user session. In such an implementation, the numberof render units allocated to a given user session may be anotherdecisional factor in deciding to increase the expiration rate and by howmuch to increase it.

In some implementations, the base token expiration rate may also bebased on financial factors. For examples, assume that two differentapplications hosted by server-side application rendering system 20 arevideo games licensed by video game publishers. One video game publishermay require 50 percent of the token revenues received by the entity thatoperates system 20, while the second video game publisher may require 20percent of token revenues. The operator of system 20 may set the baseexpiration rate to a higher value for the video game of the first videogame publisher, relative to the second video game publisher to accountfor the difference in revenue realized by the operator of system 20.

As discussed herein, a token management process may monitor operation ofserver-side application rendering system 20 relative to one of more ofCPU utilization, GPU utilization and bandwidth utilization—both in theaggregate and with respect to individual user sessions. The tokenmanagement process may also monitor the available resources of system 20and adjust the expiration rate as servers 22 come on line and go offline. CPU utilization can be accomplished by querying operating systemfacilities that monitor CPU utilization relative to applicationprocesses or threads. Bandwidth utilization may be monitored by servers22 or a remote device by classifying packets that egress from theservers 22 based on user sessions. GPU utilization may be tracked in asimilar manner to CPU utilization; provided that similar monitoringfacilities exist. If no such facilities exist, GPU utilization can beestimated based on CPU utilization, session quality settings andapplication type. In another implementation, GPU utilization can bemonitored by implementing a low priority process that, when no otherprocesses are executed, would consume all the resources of a given GPU.When other processes are executed, some of the GPU resources areconsumed by such other, higher priority processes. By monitoring theoutput of this low priority process and comparing it to a baselineoutput, a GPU load can be determined.

The token management process may increase the token expiration rates fordifferent users as the resources of server-side application renderingsystem 20 become increasingly taxed. In one particular implementation,for example, server-side application rendering system 20 may expiretokens for all users at a base expiration rate while the aggregate loadlies below a threshold—e.g., where the system 20 has more thansufficient resources to service new users and user sessions withoutcomprising the existing user sessions of other users. An example usecase may be an off-peak hours scenario, where there are lower numbers ofusers during unpopular hours. As additional users login to server-sideapplication rendering system 20 and aggregate load crosses a threshold,the token management process may begin to analyze the load attributableto individual user sessions and increase the expiration rate forindividual user sessions that exhibit higher load relative to the meanor some other value. The token management process may also increase theexpiration rate for all user sessions during peak hours with the amountof the increase depending on the individual load characteristics ofrespective user sessions.

FIGS. 3 and 4 illustrate an example process flow and method formodulating a token expiration rate in a token-based billing model forserver-side rendered application services. The operations illustrated inFIGS. 3 and 4 are described with reference to the network environmentillustrated in FIG. 1 . In one implementation, the expiration of tokensis handled by a process associated with the application instance or usersession. A separate token management process may monitor operation ofthe system 20 and adjust the expiration rates for the user sessionsindividually or in the aggregate.

FIG. 3 is a state diagram illustrating how server-side applicationrendering system 20 may transition between a baseline expiration ratemode 302 and an expiration rate adjustment mode 304. As FIG. 3illustrates, the token management process transitions from the baselineexpiration rate mode 302 to the expiration rate adjustment mode 304 inresponse to an event E1, and transitions from the expiration rateadjustment mode 304 to the baseline expiration rate mode 302 in responseto an event E2. Event E1 may be triggered by the observed aggregate loadon server-side application rendering system 20 crossing a baselinethreshold value, while event E2 may be the observed aggregate loadfalling below the baseline threshold value. In other implementations, E1and E2 can be based on temporal considerations—especially forembodiments where a given server-side application rendering system 20 isconfigured to serve a particular geographic area that may span only alimited number of time zones. For example, E1 and E2 may be triggered inresponse to time-of-day setting. For example, E1 and E2 may beconfigured to implement a billing model where tokens expire more quicklyduring administratively determined peak or premium hours relative tonon-peak or non-premium hours. In some implementations, a transitionfrom mode 302 to mode 304 may cause an across-the-board increase in thetoken expiration of all active user sessions.

FIG. 4 illustrates an example method implemented by a token managementprocess during the expiration rate adjustment mode 304. In this mode304, the token management process periodically analyzes the loadassociated with individual user sessions (310, 312) and determines thetoken expiration rate for a given user session. If token managementprocess determines to increase the token expiration rate for a usersession (314), the token management process may cause a notification tobe transmitted to the user (316). In one implementation, the usernotification may be in the form of a pop-up notification that links to auser settings interface that allows the user to adjust one or moresettings that may reduce the load associated with the user's session andtherefore decrease the token expiration rate for that user.

The logic that the token management process utilizes to determine theexpiration rate for a given user session can vary considerably toachieve a variety of different billing and charging models. Theexpiration rate determination logic can consider a number of factors,such as time of day, bandwidth utilization, CPU utilization, GPUutilization, and application type. The evaluations of many of thesefactors can be made in comparison with aggregated values (such asaverages and means) across other user sessions and/or to variousthreshold values. In addition, the determination may include weightingsof one or more of the individual factors in a component ormulti-factored algorithm for determining the expiration rate. Inaddition, the application type may be relevant to load considerations,as different applications may have different resource allocationrequirements. For example, some video game applications may requirelarger allocations of CPU resources for execution. In addition, someapplications may be more popular than other applications. Accordingly,application type can be used to account for either or both of theresource requirements for a given user session and the popularity of theapplication which relates to the number of user sessions related to thatapplication. For example, the expiration rate can be set higher for anewly released video game relative to other video games. In someimplementations involving rendering point clouds or voxels, the cost andexpiration rate can be mapped to the number of intersections of rays topoint objects in a scene. This can be applied to games that are entirelyor partially ray-traced including graphics rendering and physicsoperations.

As discussed above, a user, either in response to a notification orgenerally, may adjust one or more session quality settings that affectthe load attributable to a given user session. The session qualitysettings may include, but are not limited to display settings (such asresolution and screen size), frame rate, video quality (tied tocompression settings, for example, such as target bit rate). Asdiscussed above, the user's choice of application type may affect thebaseline resources, such as CPU/GPU/RAM resources, required to deliverthe application. For example, a user, confronted with a notification ofan increase in the token expiration rate, may decide to adjust theresolution or screen size to consume less render units, and/or opt formore aggressive compression to reduce bandwidth consumption. As anadditional example, a user may opt to turn off P-frame encoding toreceive only I-frames, which lowers the computational load to compressthe video sequence, but increases bandwidth usage (which depending onthe region may be relative small part of the cost of delivering theservice). In addition, the user may allow server-side applicationrendering system 20 to automatically determine the user's sessionquality settings to minimize the token expiration rate. Additionalsettings may include motion blur (e.g., setting this on or off to reducecomputational resources required to generate video frames), color or bitdepth, as well as other rendering parameters such as ray tracingparameters, lighting quality and shading.

In addition, the user interface that is provided to the user normallywithout any such notification may include a graphical indicator, such asa status bar, that indicates the number of remaining tokens and/or theamount of time left on the system at the current expiration rate. Inaddition, the interface may include text or graphics that indicate thecurrent rate of token expiration. In addition, the user interface mayallow the user to adjust the session quality controls discussed above,using keyboard, mouse and/or joystick inputs (or be presented with apop-up dashboard). With the visual indicators of token amount andexpiration rate, the system 20 provides the user with visual feedback ofthe effects of his or her adjustments on the rate of token expiration.For example, a user interface may include on/off controls, slider bars,pull-down menus, and the like corresponding to one or more of thesession quality settings discussed above. In addition, the system 20 canbe configured to adjust the session quality settings in real-time as theuser adjusts the controls so that the user may visually perceive theeffect of the adjustments thereby achieving a continuous feedback loop.

In another implementation, the system 20 may allow a user to controltoken expiration by specifying overall budget parameters. For example,the system 20 may allow a user to specify a token budget (i.e., amaximum number of tokens he or she desires to spend during a session orsome other time period) and either a maximum data size transfer budget(the number of bytes of data transfer) or a maximum time budget (theamount of time the user would like the token budget to last). In oneimplementation, the system 20 also allows the user to indicate othersession quality parameters that the user would like to fix, such asscreen size, render quality, and the like. The system 20 can compute oneor more session quality parameters based on the user settings and thecost considerations described herein. For example, the system 20 cancompute an initial maximum bit rate for the session based on the tokenbudget and transfer size or time budget. As the session progresses, thesystem 20 can periodically modulate the initial bit rate based on thenumber of remaining tokens and the remaining time or data transferbudget. The maximum bit rate can be reset based on the historicalconsumption during the session and the remaining number of tokens. Forexample, at points where a user has consumed half of its remainingtokens, the system 20 may reset one or more session quality settings.

A variety of factors can be considered when determining an expirationrate—whether the base or dynamically adjusted rate. The cost or loadassociated with rendered application output provided by system 20 maydepend on how the output is to be rendered by the server system,including, for example, the resolution or the size of the image, thequality of the image, the graphical operations to be applied to theimage, the amount of rendering to be performed by the server, etc., anduser specifications on how the resulting image is to be delivered to theclient device, including, for example, the bandwidth for transmittingthe image, the compression and encryption to be applied to the image,etc.

In particular embodiments, a video sequence may contain one or moreframes, i.e., digital images, and each digital image, in its raw form,is a collection of pixels. A pixel is the smallest item of informationin a digital image. Typically, pixels are arranged in a 2D grid, and arerepresented using dots, squares, or rectangles. When displayed, eachpixel in a digital image usually corresponds to a dot on the displayscreen. The content of the individual digital images may vary greatly.Some images may include graphical objects while other images may includetext. In fact, the pixels in a digital image may represent anycombination of graphical and textual objects.

The system 20 may determine a token expiration rate based on the costfor rendering and delivering the video sequence based on variousfactors, including, for example, factors relating to the rendering ofthe frames of the video sequence and factors relating to delivering thevideo sequence. In particular embodiments, the token expiration factorsmay be organized into two categories: those factors relating to therendering of an image and those factors relating to the delivering of animage by a server system. The factors relating to the rendering of animage by a server system may be collectively referred to as “renderingfactors.” The factors relating to the delivering of an image by a serversystem may be collectively referred to as “delivering factors.” Thus,the charges, e.g., the token amount and the rate at which the tokenexpires, for providing server-side rendering services may be based oneither the rendering factors, or the delivering factors, or acombination of the two categories. Several examples of the renderingfactors and the delivering factors are described below. However, therendering factors and the delivering factors are not limited to theexamples provided herewith. In addition, the expiration factors are notlimited to only theses two categories of factors.

For example, a sever typically remains in continuous operation even whenit is not actively rendering and delivering application output. Allelectronic devices require power supply while in operation. In addition,hardware tends to wear down after some period of time and software needsto be upgraded periodically. The billing model may take these factorsinto consideration as well. One factor may be related to the electricitycost of maintaining the server in continuous operation. One factor maybe related to the cost of wear and tear, replacing, and upgrading of thehardware. One factor may be related the cost of replacing or upgradingthe software.

In particular embodiments, rendering factors relate to the rendering ofa video sequence, and more specifically, to the rendering of theindividual frames in the video sequence by a server system. Each frameis in fact an image. There are many operations that may be appliedduring the rendering of an image. The image may be rendered based on thespecifications specified by the user requesting the image or originatedfrom the server performing the rendering.

One characteristic of an image is its resolution, which represents theamount of detail the image holds. An image's resolution may be measuredin different ways. In particular embodiments, a digital image'sresolution is measured by the number of pixels in the image. Higherresolution images have relatively more number of pixels, while lowerresolution images have relatively less number of pixels. One renderingfactor may be related to the resolution of the resulting image. Higherresolution images often require relatively more processing effort torender and thus may be more expensive, while lower resolution imagesusually require relatively less processing effort to render and thus maybe less expensive. The user requesting the video sequence may specifythe resolution of the resulting video frames based on the client devicethe user uses to display the image. If the client device is capable ofdisplaying high resolution images, the user may be willing to pay theextra amount to have the server render the video frames at a higherresolution. On the other hand, if the client device is only able todisplay low resolution images, then the user may choose to have theserver render the video frames at a lower resolution and save some ofthe rendering cost.

Some video sequences or images may be compressed or encoded. Toefficiently deliver the video sequence, a video code may be used—such asH.264 or some other codec. Thus, to deliver the individual frames of avideo may require that the video file be compressed. One renderingfactor may be related to the CPU and/or GPU resources for compressingthe video sequence. Relatively more complex compression effort mayresult in higher per-pixel cost, while relatively less complexcompression effort may result in lower per-pixel cost. In addition, onecost factor may consider a determination as to whether the same GPU thatrenders the video can be used to compress the resulting video stream. Ifso, PCI bus consumption can be significantly reduced. U.S. applicationSer. No. 12/579,300, which is incorporated by reference herein,discloses compression of video streams on GPUs.

Some 2D images may be rendered from 3D graphics, such as in the case ofhigh-end computer games. Ray tracing is a popular technique used torender 2D images from 3D graphics from a particular view point orperspective. The resulting 2D image is generated by tracing the paths oflight through pixels in an image plane. In particular embodiments, thequality of the resulting 2D images depends on the number of light paths,i.e., rays, used in the ray tracing process. Since ray tracing iscomputationally intensive, the number of bounces (the greater number ofrays traced for specular and glossy reflections), the more processingresources are required. One rendering factor may be related to thenumber of rays used in generating the resulting 2D image using raytracing, with greater number of bounces resulting in higher per-pixelcost and vice versa. In other implementations, a user may select whetherpath tracing, pure rasterization or direct lighting is used, as such achoice can affect the cost of rendering and the rate at which tokensexpire.

Light and shadow are often used to give the illusion of 3D in a 2Dimage. The number of light sources, the reflections of the light sourcesfrom the objects in a scene, and the depth, shape, detail, etc. of theshadow areas all contribute to the quality, e.g., realism, of an imageas well as affect the cost of rendering the image. One rendering factormay be related to the number of light sources in a scene represented bythe image. One rendering factor may be related to the number and qualityof the reflections and tin a scene represented by the image. Onerendering factor may be related to the number and quality of the shadowareas in a scene represented by the image. In particular embodiments,greater number of light sources and shadow areas require more processingpower and thus result in higher per-pixel cost.

Real objects have textures. To imitate real objects, objects representedin a digital image may also have textured surfaces. The surfaces of theobjects may be smooth or rough, shining or dim, etc. One renderingfactor may be related to the texture of the objects in a scenerepresented by the image with more complex texture algorithms resultingin higher per-pixel cost and vice versa.

Anti-aliasing is a technique used in digital image processing tominimize the distortion artifacts known as aliasing. For example, ascene represented by an image may be first created at a resolution muchhigher, e.g., 4 times, 8 times, or 16 times, of the resulting image toinclude more details. Then the higher resolution image may be scaleddown with anti-aliasing to obtain the final resulting image. In anotherexample, anti-aliasing is frequently applied when representing texts.One rendering factor may be related whether to apply anti-aliasing andhow much anti-aliasing is applied when resizing an image during therendering process. In one implementation, the system 20 allows a user toselect from different aliasing modes—including full scene anti-aliasing,edge anti-aliasing, and hardware-based antialising modes—in order toadjust token expiration.

Motion may also be depicted using various techniques. Motion isimportant to help create certain illusions and effects for animations,games, videos where a sequence of images, i.e., frames, togetherrepresent moving event rather than a static scene. To depict motionrealistically, physical laws are taken into consideration. For example,gravity, momentum, acceleration, deceleration, etc. all exert differenteffects on the objects represented in the images. One rendering factormay be related to incorporating various types of physical effects intothe rendered images. In particular embodiments, the user may specify theindividual rendering techniques to be applied during the renderingprocess or the quality of the resulting images and is then chargedaccordingly. For example, the system 20 may allow a user to select orspecify a motion blur quality and/or an amount of samples used tocompute motion blur.

In a 3D scene, from a particular view point, some objects may be infront of other objects. The object behind another object may becompletely or partially obscured by the other object in front of it.When rendering a 3D scene into a 2D image, the 2D image represents the3D scene from a particular view point. The rendering process needs todetermine, with respect to the specific point of view, which object isin the front and which object is at the back. In other words, theobjects in the scene are at different layers with respect to the viewpoint. In addition, the rendering process determines how much of theobject at the back is obscured by the object in the front. One renderingfactor may be related depth queued culling—the layering effect in thescene represented by the rendered image.

For applications such as animations, games, videos, etc., multiple stillimages together create the illusion of motion. Each still image in asequence is typically referred to as a frame. There needs to besufficient number of frames displayed each second to fool the human eyesinto seeing moving objects. Typically, videos are displayed atapproximately 30 frames per second. High-quality games are displayed atapproximately 60 frames per second. The higher the quality, the moreframes need to be rendered. One rendering factor may be related thenumber of frames generated each second. In addition, between twoconsecutive, there may be minor differences, especially in thebackground areas. Thus, from one frame to the next, the server may onlyneed to process the differences between the two frames. One renderingfactor may be related to the amount of new information the server needsto process for each frame.

It takes resources, e.g., processors, memory, etc., to execute thevarious applications and graphical operations involved in the renderingof the application output. Moreover, it takes time to perform theseapplication and graphical operations. In other words, it takes time forthe server to execute application processes and render each image of avideo sequence. In general, the more resources devoted to the executionsof the various application operations, the less time it takes to renderthe image. Some users may need to have their images rendered as quicklyas possible, while other users may be willing to experience some lagtime. One rendering factor may be related to the amount of resourcesdevoted to the application and rendering process, with relatively moreresources resulting in higher cost and vice versa. The individual usersmay have the option to specify the amount of resources to be used torender the application output. Similarly, one rendering factor may berelated to the amount of time or processing cycles taken to render eachimage, with relatively less time resulting in higher per-pixel cost andvice versa. The individual users may have the option to specify howquickly they wish to receive the resulting images.

Some client devices may have more resources in terms of processor poweror memory capacity than others. The users at these client devices maychoose to have the server only process the video frames partially anddeliver the data that enable their client devices to generate the finalresulting images. Instead of paying for the complete rendering of thevideo frames, these users may wish to only pay for the server to performcertain intermediate operations, such as the highly computationalintensive operations. The server delivers the intermediate results aswell as any additional data needed to the client devices and the clientdevices finish the rendering processes based on the intermediate resultsand additional data received from the server. One rendering factor maybe related to the percentage of rendering process to be performed by theserver, with higher percentage resulting in higher per-pixel cost andvice versa.

After a server has rendered a video sequence, the video sequence needsto be delivered to a client. In particular embodiments, deliveringfactors relate to the delivering of a video sequence to a client deviceby a server system. There are many operations that may be applied duringthe delivering of a video sequence. Similar to the rendering of thevideo sequence, the video sequence may be delivered to the client basedon the specifications specified by the user requesting the videosequence or originated from the server.

Bandwidth represents the rate at which data are transmitted, and isusually expressed in bits per second. The higher bandwidth is used, thefaster the data is transmitted. One delivering factor may be related tothe bandwidth used to deliver the resulting video sequence to the clientwith higher bandwidth resulting in higher per-pixel cost and vice versa.The user may optionally specify the bandwidth used to deliver the imagesto the user's client device.

Compression techniques may be used to decrease the size of the videodata. However, compression requires additional processing. Onedelivering factor may be related to the percentage of compression to beapplied to the resulting video sequence as well as the individual framesin the video sequence before delivering the video sequence to theclient. Higher compression ratio generally results in smaller amount ofvideo data but may result in higher per-pixel cost. Lower compressionratio generally results in larger amount of video data but may result inlower per-pixel cost. If the user does not require any compression, thenthe user is not charged for the compression cost.

Data transmitted over a network may be encrypted to provide securityprotection. Sometimes, a user may request that the server encrypt therendered video frames before delivering them to the user's clientdevice. One delivering factor may be related to the encryption of theresulting video sequence. If the user request that the video data beencrypted, then the cost of the encryption is added to the per-pixelcost.

In some implementations, the system 20 may also support an offline tokenconsumption process for non-realtime applications. For example, a userdesiring to submit a video rendering job to the system 20 can select anoffline rendering option and submit the application and job to thesystem 20. The system 20 can queue the job and process the applicationand data to create the video during off-peak periods when resources areavailable.

The methods described above may be implemented as computer softwareusing computer-readable instructions and physically stored incomputer-readable medium. For example, token management process may beimplemented as computer software that may be executed on server 22. A“computer-readable medium” as used herein may be any non-transitorymedium that can contain, store, communicate, propagate, or transport theprogram for use by or in connection with the instruction executionsystem, apparatus, system or device. The computer readable medium maybe, by way of example only but not by limitation, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, system, device, propagation medium, or computer memory.

The computer software may be encoded using any suitable computerlanguages, including future programming languages. Different programmingtechniques can be employed, such as, for example, procedural or objectoriented. The software instructions may be executed on various types ofcomputers, including single or multiple processor devices.

Embodiments of the present disclosure may be implemented by using aprogrammed general purpose digital computer, by using applicationspecific integrated circuits, programmable logic devices, fieldprogrammable gate arrays, optical, chemical, biological, quantum ornano-engineered systems, components and mechanisms may be used. Ingeneral, the functions of the present disclosure can be achieved by anymeans as is known in the art. Distributed or networked systems,components and circuits can be used. Communication, or transfer, of datamay be wired, wireless, or by any other means.

For example, FIG. 2 illustrates an example computer system 200 suitablefor implementing embodiments of the present disclosure. The componentsshown in FIG. 2 for computer system 200 are exemplary in nature and arenot intended to suggest any limitation as to the scope of use orfunctionality of the computer software implementing embodiments of thepresent disclosure. Neither should the configuration of components beinterpreted as having any dependency or requirement relating to any oneor combination of components illustrated in the exemplary embodiment ofa computer system. Computer system 200 may have many physical formsincluding an integrated circuit, a printed circuit board, a smallhandheld device (such as a mobile telephone or PDA), a personal computeror a super computer.

A “processor,” “process,” or “act” includes any human, hardware and/orsoftware system, mechanism or component that processes data, signals orother information. A processor can include a system with ageneral-purpose central processing unit, multiple processing units,dedicated circuitry for achieving functionality, or other systems.Processing need not be limited to a geographic location, or havetemporal limitations. For example, a processor can perform its functionsin “real time,” “offline,” in a “batch mode,” etc. Portions ofprocessing can be performed at different times and at differentlocations, by different (or the same) processing systems.

Although the acts, operations or computations disclosed herein may bepresented in a specific order, this order may be changed in differentembodiments. In addition, the various acts disclosed herein may berepeated one or more times using any suitable order. In someembodiments, multiple acts described as sequential in this disclosurecan be performed at the same time. The sequence of operations describedherein can be interrupted, suspended, or otherwise controlled by anotherprocess, such as an operating system, kernel, etc. The acts can operatein an operating system environment or as stand-alone routines occupyingall, or a substantial part, of the system processing.

Reference throughout the present disclosure to “particular embodiment,”“example embodiment,” “illustrated embodiment,” “some embodiments,”“various embodiments,” “one embodiment,” or “an embodiment” means that aparticular feature, structure, or characteristic described in connectionwith the embodiment is included in at least one embodiment of thepresent disclosure and not necessarily in all embodiments. Thus,respective appearances of the phrases “in a particular embodiment,” “inone embodiment,” “in some embodiments,” or “in various embodiments” invarious places throughout this specification are not necessarilyreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics of any specific embodiment of the presentdisclosure may be combined in any suitable manner with one or more otherembodiments. It is to be understood that other variations andmodifications of the embodiments of the present disclosure described andillustrated herein are possible in light of the teachings herein and areto be considered as part of the spirit and scope of the presentdisclosure.

It will also be appreciated that one or more of the elements depicted inFIGS. 1 through 3 can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise. Additionally, theterm “or” as used herein is generally intended to mean “and/or” unlessotherwise indicated. Combinations of components or steps will also beconsidered as being noted, where terminology is foreseen as renderingthe ability to separate or combine is unclear.

The present disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsherein that a person having ordinary skill in the art would comprehend.Similarly, where appropriate, the appended claims encompass all changes,substitutions, variations, alterations, and modifications to the exampleembodiments herein that a person having ordinary skill in the art wouldcomprehend.

What is claimed is:
 1. A method comprising: receiving, at a remotecomputing device, a token associated with a user computing device, thetoken allowing access to processing resources of the remote computingdevice, the processing resources comprising hardware operating anexecutable application running on the remote computing device;receiving, at the remote computing device, one or more parameterscontrolling an operation of the processing resources, the processingresources allocated for executing the executable application in responseto a request from the user computing device; operating, by the remotecomputing device, the processing resources based on the token and theparameters, the operating the processing resources comprising at leastpartially rendering at least one frame of image data generated by theapplication running on the remote computing device; and transmitting, bythe remote computing device to the user computing device, the at leastpartially rendered frame of image data.
 2. The method of claim 1, thereceiving the token comprising receiving a digital object digitallysigned using a cryptographic function.
 3. The method of claim 1, thereceiving the token comprising receiving a token stored in a centralizeddatabase.
 4. The method of claim 1, further comprising: determining thatthe token has expired; and halting the operating of the processingresources when the token has expired.
 5. The method of claim 1, theoperating the processing resources comprising operating a graphicsprocessing unit (GPU) of the remote computing device.
 6. The method ofclaim 1, the at least partially rendering at least one frame of imagedata comprising at least partially rendering a plurality of frames, theplurality frames comprising a video.
 7. The method of claim 1, thereceiving the one or more parameters comprising receiving a parameterselected from the group consisting of an image resolution value, asample number, a color depth, and a bit depth.
 8. A non-transitorycomputer-readable storage medium for tangibly storing computer programinstructions capable of being executed by a computer processor, thecomputer program instructions defining steps of: receiving a tokenassociated with a user computing device, the token allowing access toprocessing resources accessible by the computer processor, theprocessing resources comprising hardware operating an executableapplication running on the computer processor; receiving one or moreparameters controlling an operation of the processing resources, theprocessing resources allocated for executing the executable applicationin response to a request from the user computing device; operating theprocessing resources based on the token and the parameters, theoperating the processing resources comprising at least partiallyrendering at least one frame of image data generated by the applicationrunning on the computer processor; and transmitting, to the usercomputing device, the at least one partially rendered frame of imagedata.
 9. The non-transitory computer-readable storage medium of claim 8,the receiving the token comprising receiving a digital object digitallysigned using a cryptographic function.
 10. The non-transitorycomputer-readable storage medium of claim 8, the receiving the tokencomprising receiving a token stored in a centralized database.
 11. Thenon-transitory computer-readable storage medium of claim 8, the computerprogram instructions further defining the steps of: determining that thetoken has expired; and halting the operating of the processing resourceswhen the token has expired.
 12. The non-transitory computer-readablestorage medium of claim 8, the operating the processing resourcescomprising operating a graphics processing unit (GPU) accessible by thecomputer processor.
 13. The non-transitory computer-readable storagemedium of claim 8, the at least partially rendering at least one frameof image data comprising at least partially rendering a plurality offrames, the plurality frames comprising a video.
 14. The non-transitorycomputer-readable storage medium of claim 8, the receiving the one ormore parameters comprising receiving a parameter selected from the groupconsisting of an image resolution value, a sample number, a color depth,and a bit depth.
 15. An apparatus comprising: a processor; one or moreprocessing resources accessible by the processor; and a non-transitorystorage medium for tangibly storing thereon program logic for executionby the processor, the stored program logic comprising: logic, executedby the processor, for receiving a token associated with a user computingdevice, the token allowing access to the processing resources, theprocessing resources comprising hardware operating an executableapplication running on the processor, the processing resources allocatedfor executing the executable application in response to a request fromthe user computing device, logic, executed by the processor, forreceiving one or more parameters controlling an operation of theprocessing resources, logic, executed by the processor, for operatingthe processing resources based on the token and the parameters, theoperating the processing resources comprising at least partiallyrendering at least one frame of image data generated by the applicationrunning on the processor, and logic, executed by the processor, fortransmitting, to the user computing device, the at least one partiallyrendered frame of image data.
 16. The apparatus of claim 15, the logicfor receiving the token comprising logic, executed by the processor, forreceiving a digital object digitally signed using a cryptographicfunction.
 17. The apparatus of claim 15, the stored program logicfurther comprising: logic, executed by the processor, for determiningthat the token has expired; and logic, executed by the processor, forhalting the operating of the processing resources when the token hasexpired.
 18. The apparatus of claim 15, the processing resourcescomprising at least one graphics processing unit (GPU).
 19. Theapparatus of claim 15, the logic for least partially rendering at leastone frame of image data comprising logic, executed by the processor, forat least partially rendering a plurality of frames, the plurality framescomprising a video.
 20. The apparatus of claim 15, the logic forreceiving the one or more parameters comprising logic, executed by theprocessor, for receiving a parameter selected from the group consistingof an image resolution value, a sample number, a color depth, and a bitdepth.